{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4fef01c7",
   "metadata": {},
   "source": [
    "# 自定义数据集\n",
    "mindspore.dataset和mindvision.classification.dataset中提供了部分常用数据集的接口，对于其中没有的数据集，我们可以使用`mindspore.dataset.GeneratorDataset`接口来实现自定义数据集加载。\n",
    "\n",
    "使用这个方法有两种方式，一种是`自定义数据集类`，一种是`自定义数据集`。一般使用`自定义数据集类的方法`。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23922547",
   "metadata": {},
   "source": [
    "## 自定义数据集类\n",
    "自定义数据集类中需包含三个方法：\n",
    "- `__init__`：定义数据初始化等操作，在实例化数据集对象时被调用。\n",
    "- `__getitem__`：定义该函数后可使其支持随机访问，能够根据给定的索引值`index`，获取数据集中的数据并返回。数据返回值类型是由NumPy数组组成的Tuple。\n",
    "- `__len__`：返回数据集的样本数量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "02b4ae76",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import mindspore.dataset as ds\n",
    "\n",
    "np.random.seed(1)\n",
    "\n",
    "class MyDataset:\n",
    "    \"\"\"自定义数据集类\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        self.data = np.random.sample((6, 2))  # 自定义数据\n",
    "        self.label = np.random.sample((6, 1))  # 自定义标签\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        return self.data[index], self.label[index]\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.data) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aea9bf99",
   "metadata": {},
   "source": [
    "自定义数据集后，我们可以使用`GeneratorDataset`接口来加载数据集，根据任务的不同，我们可以指定单标签数据集还是多标签数据集。\n",
    "\n",
    "### 生成单标签数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9c62174e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(61148:60472,MainProcess):2022-10-26-23:09:59.661.505 [mindspore\\dataset\\engine\\datasets_user_defined.py:656] Python multiprocessing is not supported on Windows platform.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data:[0.41702, 0.72032], label:[0.20445]\n",
      "data:[0.00011, 0.30233], label:[0.87812]\n",
      "data:[0.14676, 0.09234], label:[0.02739]\n",
      "data:[0.18626, 0.34556], label:[0.67047]\n",
      "data:[0.39677, 0.53882], label:[0.41730]\n",
      "data:[0.41919, 0.68522], label:[0.55869]\n",
      "data size: 6\n"
     ]
    }
   ],
   "source": [
    "# 实例化数据集类\n",
    "dataset_generator = MyDataset()\n",
    "dataset = ds.GeneratorDataset(dataset_generator, [\"data\", \"label\"], shuffle=False)\n",
    "\n",
    "# 迭代访问数据集\n",
    "for data in dataset.create_dict_iterator():\n",
    "    data1 = data['data'].asnumpy()\n",
    "    label1 = data['label'].asnumpy()\n",
    "    print(f'data:[{data1[0]:7.5f}, {data1[1]:7.5f}], label:[{label1[0]:7.5f}]')\n",
    "\n",
    "# 打印数据条数\n",
    "print(\"data size:\", dataset.get_dataset_size())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d877b60e",
   "metadata": {},
   "source": [
    "### 生成多标签数据集\n",
    "我们只需要在`__init__`方法中多初始化label就能实现多标签数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "0e906b5a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(61148:60472,MainProcess):2022-10-26-23:17:48.588.888 [mindspore\\dataset\\engine\\datasets_user_defined.py:656] Python multiprocessing is not supported on Windows platform.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data:[0.87639, 0.89461]   label1:[0.42111] label2:[0.68650]\n",
      "data:[0.14039, 0.19810]   label1:[0.16983] label2:[0.53317]\n",
      "data:[0.08504, 0.03905]   label1:[0.95789] label2:[0.83463]\n",
      "data:[0.80074, 0.96826]   label1:[0.87814] label2:[0.69188]\n",
      "data:[0.31342, 0.69232]   label1:[0.09835] label2:[0.31552]\n",
      "data size: 5\n"
     ]
    }
   ],
   "source": [
    "class MyDataset:\n",
    "    \"\"\"自定义多标签数据集\"\"\"\n",
    "    def __init__(self):\n",
    "        self.data = np.random.sample((5, 2))\n",
    "        self.label1 = np.random.sample((5, 1))\n",
    "        self.label2 = np.random.sample((5, 1))\n",
    "    \n",
    "    def __getitem__(self, index):\n",
    "        return self.data[index], self.label1[index], self.label2[index]\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.data)\n",
    "\n",
    "dataset_generator = MyDataset()\n",
    "dataset = ds.GeneratorDataset(dataset_generator, [\"data\", \"label1\", \"label2\"])\n",
    "\n",
    "for data in dataset.create_dict_iterator():\n",
    "    print(\"data:[{:7.5f},\".format(data['data'].asnumpy()[0]),\n",
    "          \"{:7.5f}]  \".format(data['data'].asnumpy()[1]),\n",
    "          \"label1:[{:7.5f}]\".format(data['label1'].asnumpy()[0]),\n",
    "          \"label2:[{:7.5f}]\".format(data['label2'].asnumpy()[0]))\n",
    "\n",
    "print(\"data size:\", dataset.get_dataset_size())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mindspore",
   "language": "python",
   "name": "mindvision"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
